[Spark-4512] [SQL] Unresolved Attribute Exception in Sort By #3386

chenghao-intel · 2014-11-20T14:59:17Z

It will cause exception while do query like:
SELECT key+key FROM src sort by value;

SparkQA · 2014-11-20T15:02:52Z

Test build #23673 has started for PR 3386 at commit 503b263.

This patch merges cleanly.

SparkQA · 2014-11-20T15:14:14Z

Test build #23673 has finished for PR 3386 at commit 503b263.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Sort(

AmplabJenkins · 2014-11-20T15:14:17Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23673/
Test FAILed.

SparkQA · 2014-11-25T08:37:37Z

Test build #23832 has started for PR 3386 at commit e0047a0.

This patch merges cleanly.

SparkQA · 2014-11-25T09:14:06Z

Test build #23832 has finished for PR 3386 at commit e0047a0.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-11-25T09:14:09Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23832/
Test FAILed.

SparkQA · 2014-11-25T14:35:24Z

Test build #23841 has started for PR 3386 at commit 6e720af.

This patch merges cleanly.

SparkQA · 2014-11-25T15:46:42Z

Test build #23841 has finished for PR 3386 at commit 6e720af.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Sort(

AmplabJenkins · 2014-11-25T15:46:46Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23841/
Test PASSed.

marmbrus · 2014-12-17T19:58:19Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala

              Project(projectList ++ missingInProject, child)))
        } else {
          logDebug(s"Failed to find $missingInProject in ${p.output.mkString(", ")}")
          s // Nothing we can do here. Return original plan.
        }
-      case s @ Sort(ordering, a @ Aggregate(grouping, aggs, child)) if !s.resolved && a.resolved =>
+      case s @ Sort(ordering, global, a @ Aggregate(grouping, aggs, child))
+        if !s.resolved && a.resolved =>


Indent 4 spaces.

marmbrus · 2014-12-17T20:00:15Z

Sorry for the delay! This looks great. Two minor comments then LGTM.

Add `sort by` support for both DSL & SqlParser. This PR is relevant with #3386, either one merged, will cause the other rebased. Author: Cheng Hao <[email protected]> Closes #3481 from chenghao-intel/sortby and squashes the following commits: 041004f [Cheng Hao] Add sort by for DSL & SimpleSqlParser

SparkQA · 2014-12-19T01:32:34Z

Test build #24615 has started for PR 3386 at commit b4bde6a.

This patch merges cleanly.

SparkQA · 2014-12-19T02:42:45Z

Test build #24615 has finished for PR 3386 at commit b4bde6a.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Sort(

AmplabJenkins · 2014-12-19T02:42:48Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24615/
Test PASSed.

SparkQA · 2014-12-19T14:27:30Z

Test build #24646 has started for PR 3386 at commit b229fe9.

This patch merges cleanly.

SparkQA · 2014-12-19T15:37:30Z

Test build #24646 has finished for PR 3386 at commit b229fe9.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Sort(

AmplabJenkins · 2014-12-19T15:37:33Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24646/
Test PASSed.

marmbrus · 2014-12-19T18:57:23Z

sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala

@@ -238,7 +238,7 @@ class SchemaRDD(
   * {{{
   *   schemaRDD.limit(10)
   * }}}
-   * 
+   * o


Oh, yes, it's a typo, revert it. Thanks.

SparkQA · 2014-12-20T02:02:28Z

Test build #24661 has started for PR 3386 at commit aae675f.

This patch merges cleanly.

SparkQA · 2014-12-20T03:10:42Z

Test build #24661 has finished for PR 3386 at commit aae675f.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Sort(

AmplabJenkins · 2014-12-20T03:10:45Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24661/
Test PASSed.

SparkQA · 2014-12-22T01:07:31Z

Test build #24690 has started for PR 3386 at commit 38c78cc.

This patch merges cleanly.

SparkQA · 2014-12-22T02:18:20Z

Test build #24690 has finished for PR 3386 at commit 38c78cc.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class Sort(

AmplabJenkins · 2014-12-22T02:18:23Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24690/
Test PASSed.

chenghao-intel force-pushed the sort branch from 503b263 to e0047a0 Compare November 25, 2014 08:33

This was referenced Nov 26, 2014

[SPARK-4625] [SQL] Add sort by for DSL & SimpleSqlParser #3481

Closed

[SPARK-4636] [SQL] Cluster By & Distribute By should follows the Hive behavior #3496

Closed

marmbrus reviewed Dec 17, 2014
View reviewed changes

chenghao-intel changed the title ~~[Spark-4512] [SQL] Unresolved Attribute Exception in Sort By~~ [Spark-4512] [SQL] WIP: Unresolved Attribute Exception in Sort By Dec 19, 2014

chenghao-intel force-pushed the sort branch from 6e720af to b4bde6a Compare December 19, 2014 01:27

chenghao-intel force-pushed the sort branch from b4bde6a to b229fe9 Compare December 19, 2014 14:23

chenghao-intel changed the title ~~[Spark-4512] [SQL] WIP: Unresolved Attribute Exception in Sort By~~ [Spark-4512] [SQL] Unresolved Attribute Exception in Sort By Dec 19, 2014

marmbrus reviewed Dec 19, 2014
View reviewed changes

chenghao-intel added 3 commits December 21, 2014 16:55

rebase the latest master and update the SortBy unit test

fcd1d64

update the typo

7e9dd15

revert the SortPartition in SparkStrategies

38c78cc

chenghao-intel force-pushed the sort branch from aae675f to 38c78cc Compare December 22, 2014 01:04

asfgit closed this in 53f0a00 Dec 30, 2014

chenghao-intel deleted the sort branch July 2, 2015 08:41

[Spark-4512] [SQL] Unresolved Attribute Exception in Sort By #3386

[Spark-4512] [SQL] Unresolved Attribute Exception in Sort By #3386

Uh oh!

Conversation

chenghao-intel commented Nov 20, 2014

Uh oh!

SparkQA commented Nov 20, 2014

Uh oh!

SparkQA commented Nov 20, 2014

Uh oh!

AmplabJenkins commented Nov 20, 2014

Uh oh!

SparkQA commented Nov 25, 2014

Uh oh!

SparkQA commented Nov 25, 2014

Uh oh!

AmplabJenkins commented Nov 25, 2014

Uh oh!

SparkQA commented Nov 25, 2014

Uh oh!

SparkQA commented Nov 25, 2014

Uh oh!

AmplabJenkins commented Nov 25, 2014

Uh oh!

marmbrus Dec 17, 2014

Choose a reason for hiding this comment

Uh oh!

marmbrus commented Dec 17, 2014

Uh oh!

SparkQA commented Dec 19, 2014

Uh oh!

SparkQA commented Dec 19, 2014

Uh oh!

AmplabJenkins commented Dec 19, 2014

Uh oh!

SparkQA commented Dec 19, 2014

Uh oh!

SparkQA commented Dec 19, 2014

Uh oh!

AmplabJenkins commented Dec 19, 2014

Uh oh!

marmbrus Dec 19, 2014

Choose a reason for hiding this comment

Uh oh!

chenghao-intel Dec 20, 2014

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Dec 20, 2014

Uh oh!

SparkQA commented Dec 20, 2014

Uh oh!

AmplabJenkins commented Dec 20, 2014

Uh oh!

SparkQA commented Dec 22, 2014

Uh oh!

SparkQA commented Dec 22, 2014

Uh oh!

AmplabJenkins commented Dec 22, 2014

Uh oh!

Uh oh!